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Abstract 


The aim of this study is to assess the science competency of university students enrolling in the School 
of Biochemistry and Biological Sciences (first year of the courses of Biochemistry and Biotechnology at 
the Universidad Nacional del Litoral de Santa Fe, Argentina) and examine the effects of these courses 
on their competencies of Chemistry proficiency. The concept of science competency has been drawn 
from the 2006 Program for International Student Assessment (PISA). To collect the information, two ad 
hoc questionnaires were drafted with questions addressing three science sub-competencies: identifying 
scientific issues (ISI), explaining phenomena scientifically (EPS) and using scientific evidence (USE). The 
first questionnaire was designed with the aim to inquire into the initial sub-competencies of the first-year 
students (n= 84). The second one (n= 50) was designed to explore the influence of the courses on their 
developments of such sub-competencies. 

The results indicate that first-year students started with a low performance level for all the sub-competencies 
assessed but the first-year of education influenced to develop the sub-competencies identifying scientific 
issues and explaining phenomena scientifically. Performance levels on using scientific evidence decreased. 
Therefore, the findings suggested that the courses in the first year need to help students or boost this sub- 
competency to be able to draw conclusions based on evidence about science-related issues. 

Key words: chemistry, higher education, science competency. 


Introduction 


The citizens of the 21st century must analyze situations and make decisions on issues 
that have to do with either scientific knowledge or technical skills. The teaching of science 
should not be just concerned with the transmission of a series of unconnected, often obsolete 
items of knowledge; nor should the role of student bodies be to merely accrue such knowledge. 
On the contrary, a new perspective implies promoting a teaching model that helps students to 
develop more consistent, flexible and especially critical comprehension (Hodson, 2003). 
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For this reason, the teaching of science must incorporate a focus based both on students’ 
day-to-day experiences as well as on the ways in which scientific knowledge may meet the 
needs of society today. That is, the use of “context” as the basis to design the curricula and 
teaching in the classroom (Gilbert, 2006; Ultay & Calik, 2012). 

As the Nuffield Report (Osborne & Dillon, 2008) underlines, traditional science education 
is contented with transmitting a vision of science that is insufficient to meet the needs of youths, 
and incapable of preparing them to understand the role of science and technology in today’s 
world. The greatest concerns of humankind nowadays are food, health, energy, water, climate 
change; and it is clear that such problems will not be addressed nor solved without important 
scientific contributions, nor without citizens understanding them and becoming engaged (Calik 
& Coll, 2012; Calik, Turan & Coll, 2013). 

The PISA 2006 (OECD, 2006) focused its attention on Science competency. “In the 
definition proposed here, science competency is conceived as a continuum encompassing from 
the lowest to the most advanced levels of science competency”’ (OECD, 2006, p. 26). In other 
words, it is considered that individuals possess several degrees of science competency and not 
that they may or not possess it in absolute terms (Bybee, 1997). The PISA statements evidence 
a turnaround in the approach to teaching that involves all stages of education, not only the 
basic ones but the renovation of university education. The need is underlined to focus more on 
how and what the knowledge is used for. That is, the ability to solve problems is valued more 
than memorized knowledge (Bybee, 1997; Harlen, 2002). The concept of science competency 
from PISA 2006 includes three science sub-competencies: identifying scientific issues (ISI), 
explaining phenomena scientifically (EPS) and using scientific evidence (USE). 

Argentina, as a member country, has taken part together with other 56 countries from 
all over the world in the PISA 2006 study. The results achieved by Argentinian teenagers (aged 
15 years) were alarming, as for science competency they ranked 51‘ (OECD, 2008). In the 
studies of 2009 and 2012, the evolutions were negative as they achieved position no. 56 out of 
64 countries (OECD, 2010) and no. 58 out of 65 countries (OECD, 2013), respectively. Taking 
into account the problems above, we agree with Monereo et al. (2009) on the need to improve 
the educational system and in the suitability of doing so, starting precisely from the reflections 
issuing from the PISA results. 

The contents of science are fundamental to achieving scientific literacy, as are aimed 
at answering the question how is known?, versus what is known? (Bartholomew, Osborne 
& Ratcliffe, 2004).The statements regarding PISA show a shift in the approach to teaching 
that goes through all educational stages. The requirement of universities to prepare students 
in science competency is gaining more importance. In this sense, there is a growing body of 
research concerned with the identification and assessment of competencies, as they become 
an indicator of the quality of education provided (Brown & Glasner, 2003; Cano Garcia, 
2008; Vivas & Hevia, 2009; Prades & Espinar, 2010). The fact that the PISA assessments 
do not depend on the curriculum can help to establish innovative approaches to teaching. To 
investigate the development of science competency from PISA 2006 (OCDE, 2008) perspective 
in the university freshmen year could provide some guidance on the priorities over quality 
education. 

All of the above highlights the relevance acquired by training based on competencies 
and the learning context of the new curricula. Therefore, the aim of this paper is to assess 
which science competency students possess when they enroll in the School of Biochemistry 
and Biology Science (first year in the courses of Biochemistry and Biotechnology) and which 
competency they possess when they complete the first year of the above courses mentioned. 

It must be highlighted that teaching is a secondary duty for most scientific bodies at 
universities, since they generally develop teaching skills as a consequence of their teaching 
practice. Their limited training on science teaching methodologies and, in general, the little 
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involvement of university science teachers in the research into science teaching is expressed by 
Dehaan (2005). 

Given that the courses are becoming ever more specialized, research into which 
Chemistry content is more suitable for students of different age groups and courses, as well 
as which teaching methodologies prove to be the most efficient becomes ever more important 
(Gabel, 1999). Consequently, it was considered relevant to research into what is happening at 
the Argentinian higher education level, more specifically at the School of Biochemistry and 
Biology Science of the Universidad Nacional del Litoral. 

As a consequence of the above, the following questions are put forward: What is 
the science competency of university students enrolling at the School of Biochemistry and 
Biological Sciences (courses of Biochemistry and Biotechnology)? What is the competency 
they have acquired by the time they finish their first year? What are their developments of the 
sub-competencies like? Does development of science competency depend on education in the 
first year of the courses mentioned? 

The aim of this study is to assess the science competency of university students enrolling 
in the School of Biochemistry and Biological Sciences (first year of the courses of Biochemistry 
and Biotechnology at the National University of the Littoral, Argentina) and examine the effects 
of these courses on their competencies of Chemistry proficiency. 


Methodology of Research 


In this section, information about general background of research, characteristics of the 
sample, data collection and analysis, validity and reliability and procedures of the courses are 
provided. 


General Background of Research 


This descriptive study is based on the ex post facto type (non experimental). It focused on 
the analysis assessing the science competency as dependent variable of the instruction received 
in the first year courses of Biochemistry and Biotechnology without taking into account neither 
the procedures of the courses nor the teaching intervention. This research includes aspects of 
quantitative and qualitative methodologies (Cohen y Manion, 2002; Bisquerra Alzina, 2004). 

Standard statistical techniques were used. Categorical variables were analyzed by Chi- 
Square Test and the Fisher’s exact test, whenever appropriate (Agresti, 2002). All analyses were 
conducted by R (R Development Core Team, 2010). 


Subjects and Characteristics of the Sample 


The research started with a sample of 84 first-year undergraduate students and the make- 
up was as follows: 47 students of Ist year of Biochemistry (B) and 37 students of 1st year 
of Biotechnology (LB). Participating students were volunteers. The sample consisted of 84 
students 77% of whom were female and 23% were male and the average age of the subjects was 
18 years old. The students came from public (58%) and private (42%) secondary schools with 
medium socio-economic status. The selection was performed using stratified sampling, seeking 
homogeneity within each stratum and heterogeneity among strata. The number of students is 
considered adequate -greater than 10% of students enrolled, according to the recommendation 
by Colas Bravo & Buendia Eisman (1998). 

Subsequently, towards the end of that same year the research was continued with a 
sample of 50 of the same students: 27 of Biochemistry and 23 of Biotechnology. Such decrease 
in the number of subjects in the sample was due to voluntary or forced dropout from a course, 
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either temporarily or permanently. During that year, the aforesaid students enrolled, among 
others, in the following subject matters related to chemistry knowledge: General Chemistry and 
Inorganic Chemistry. This makes both groups eligible to assess their degree of preparedness 
to face the daily challenges of modern society and thus obtain a quite approximate idea of the 
competencies that they have achieved during their first year in the University course. It must 
be clarified that during the first year, students attend the same courses (chemistry, biology, labs, 
etc.) in both, Biochemistry and Biotechnology. 


Information Collection Strategy 


This section describes the information collection strategy, to which purpose two 
questionnaires on science competency have been drafted and used. 

Two different questionnaires were designed based on the definition of science competency 
in the PISA 2006 report, which may be characterized by four interrelated aspects (OECD 2002, 
2004 and 2006): skills, knowledge, areas of application and attitudes; and some tests in the 
aforesaid report were selected. Subsequently, such tests have been adapted to the university 
level and other tests of our own design have also been used to draft the questionnaires. Three 
levels of description and interpretation -macroscopic, microscopic and symbolic (Johnstone, 
1993)- of chemistry knowledge were assessed in different contexts, considered to be desirable 
from the viewpoint of academic knowledge. 

During the study, the first questionnaire contained 17 questions and was divided into two 
topics: acid rain (10) and caries (7). It was drafted at the beginning of the year to determine the 
science competency of first-year students. 

The second questionnaire consisted of 15 questions, distributed according to the same 
topics, 8 and 7 respectively. It was drafted at the end of the year, in order to explore the influence 
that teaching and learning may have had on the acquisition of such competencies. 

These topics were selected given that the areas of application (Environment and Health) 
are closely connected to the courses of Biochemistry and Biotechnology where this study 
was conducted. Each question enables to assess one of the science skills, requiring the use of 
either knowledge of science or knowledge about science. Both topics assess (through different 
questions) more than one skill and more than one field of knowledge. 


Validity and Reliability 


The internal validity in the design of the questionnaire on science competency is 
established by taking the base model of the items designed and the study validated by PISA. 
Likewise, the internal rational method has been used which enables to determine the validity of 
a given instrument, checking whether it measures that which it has been designed for, that is to 
say, in this case, whether it measures the science sub-competencies: ISI, EPS and USE. 

Cronbach’ Alpha was chosen from among the most usual methods to measure the 
degree of reliability of the instruments. The values for a = 0.72 and a = 0.75 for the first and 
second questionnaires respectively indicate that the opinion results of the 84 students with 
respect to the 17 items analyzed and of the 50 students with respect to the 15 items considered 
correlate in a highly reliable and acceptable manner. 

Therefore, in order to reach the final version of the instrument, a group of experts from 
our faculty were requested to read and consider the texts that are being assessed in each item 
and the corresponding answer keys. 
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Strategy for Information Analysis 


The categories «maximum score», «partial score» and «no score» divide the students’ 
answers into three groups, according to the skills they show when answering the question asked. 
An answer qualified with the «maximum score» does not mean that the answer is absolutely 
correct in scientific terms, but that it does have the adequate level of understanding of the topic 
for a university student who is competent in science. Answers that are less elaborated, or not so 
correct, may obtain a «partial score». Answers that are incorrect, irrelevant or lack of answer 
will be marked with the category «no score» (Abraham, Gryzybowski, Renner & Marek, 1992; 
Abraham, Williamson & Westbrook, 1994). 

The possible scoring for closed or multiple option questions varies from 0 to 2 points. 
Open questions, which may have different levels of correction, will have different scores 
ranging from 0 to | and 2 points. The appendices are an example of this. 

Due to the numerical dispersion of the scores achieved, the results are grouped into 
performance levels. For the analysis of the Total performance of science competency (TPSC), 
the baseline or average considered was 50% of the total score, if all the answers obtained the 
maximum score. That is, out of a total of 17 questions, whose maximum score is 2 points each 
adding up to 34 points (100%), the desirable score for the average level of performance was set 
at 17 points (50%). 

For the different competencies the baseline or average of 50% of the total score of each 
one of them was established, if all the answers had achieved the maximum score. That is, for: 

e Identifying scientific issues USI), for example, out of a total of 3 questions, whose 
maximum score is 2 points each and in total add up to 6 points (100%), the desirable score for 
the average level of performance was set at 3 points (50%). 

e Explaining scientific phenomena (EPS), for example, out of a total of 10 questions, 
whose maximum score is 2 points each and in total add up to 20 points (100%), the desirable 
score for the average level of performance was set at 10 points (50%). 

e Using scientific evidence (USE), for example, out of a total of 4 questions, whose 
maximum score is 2 points each and in total add up to 8 points (100%), the desirable score for 
the average level of performance was set at 4 points (50%). 

As has been stated above, it is considered that individuals possess several degrees of science 
competency and not that they may or not possess such competency in absolute terms. Therefore, 
in order to communicate the results by categories or in a continuous scale, an adaptation was 
made to the procedure of qualitative assessment and communication in percentages proposed 
by Biggs (2005, p. 236). According to this criterion, Table 1 presents the levels of science 
competency according to the range of scores obtained expressed in percentages. 


Table 1. Scoring scales. 











Category of qualification of competency ik 
Reaches objectives at the maximum level 100 to 70 
Level of competency at average level 70 to 50 
Low performance level <50 





For example, a student having achieved the following scores: TPSC=13 points; [S/=5; 
EPS=4 and USE=4; corresponds respectively to 38% of the total of TPSC; 83% of IST; 20% 
of EPS and 50% of USE. Consequently and according to Table 1, such student belongs in the 
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following levels: low in 7PSC; maximum for /S/; low in EPS and average for USE. Subsequently, 
the amount of students in each category or level was counted according to Table 1, and the 
relevant percentage of the total students was estimated for each sub-sample. 


Procedures of the Courses 


The procedures in the courses take place in traditional teaching, such as lectures, 
laboratory, workshops and tests. 

It should be pointed out that, in the first year, rigid approaches still prevail in the 
curriculums of these courses. 


Results of Research 


Table 2 contains the performance results of the science competency for each instances 
assessed, expressed as the percentage of students who achieved the different levels of 
performance. The aforesaid table expresses the following taxonomic categories described 
above: TPSC, ISI, EPS and USE. 


Table 2. Percentages of levels of science competency of first-year students in 
the courses of Biochemistry and Biotechnology. 





% First-year students 






































Biochemistry Biotechnology 
Student 
PRE-CLASS pets ei xy) PRE-CLASS Ber varia- 
Competency ventions) tion (%) Score 
TPSC 2 8 6 0 0 0 
Maximum 
ISI 17 37 20 14 22 8 level 
(% score 
USE 32 4 -28 30 9 2 me) 
TPSC 21 22 1 3 30 27 Average 
isi 43 30 13 35 26 9 level (% score 
obtained 
EPS 17 33 16 8 22 14 70 to 50) 
USE 38 26 12 38 39 1 
TPSC 77 70 x 97 70 Di 
ISI 40 33 f 51 52 1 Pow lie 
a (% SCOFE 
EPS 83 60 -23 92 78 -14 obtained 
< 50) 
USE 30 70 40 32 52 20 





The results enabled to learn the extent of student performance in science competency, 
with regards to the length of enrollment and to each one of the university courses. In effect, it 
may be observed in Table 2: 

Beginning the first year of education (pre-class): 

e TPSC: total performance in science competency. In the first years of Biochemistry and 
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Biotechnology, respectively, it was found that 77% and 97% of students are below the basic 
level set. This means that for this initial level most of the students in these courses, even though 
having passed the exam to access University, lack a sound base of knowledge and skills from 
their school years. 

e ISI: identifving scientific issues. Biochemistry and Biotechnology have, respectively, 
40% and 51% of students below the basic level set. 

e EPS: explaining phenomena scientifically, this is probably the sub-competency that 
is most worked with in the classroom as it is based on the application of scientific concepts. 
However, this is generally the competency with the lowest scores. The first years of Biochemistry 
and Biotechnology have, respectively, 83% and 92% of students below the basic level set. Both 
have 0% in the higher level, and Biochemistry (17%) shows twice as much as Biotechnology 
(8%) in the average level. First-year students of Biochemistry score better than those of 
Biotechnology to start with. 

e USE: using scientific evidence, a competency focusing on evidence and argumentation, 
shows similar scores for both courses. First-year students of Biochemistry and Biotechnology 
are, respectively, 30% and 32% below the basic level. The highest percentages are in the average 
level (38%). That is, this is the swb-competency with the best scores in this instance. 

At the end of the first year, after one year of education (post-class): 

e7PSC: Biochemistry course shows a slight increase in the maximum level of total 
performance of scientific competency (8%). Most students of Biochemistry and Biotechnology 
are grouped into the average (22%-30%) and low (70%-70%) levels. Considering that these 
students have undergone one year of higher education, these findings are remarkably low. 

e IST: identifying scientific issues is the sub-competency that shows the greatest increase 
in the maximum level for both courses (37%-22%). The variation in Biochemistry is as follows: 
decrease by 7% and 13 % in the low and average levels respectively, and increase by 20 % in 
the high level. The variation in Biotechnology is: increase by 1 % and 8 % in the low and high 
levels, respectively, and decrease by 9 % in the average level. Therefore, the improvement 
is more significant in Biochemistry as in Biotechnology the score for the low level remains 
practically the same (52%). 

e EPS: in both courses, explaining phenomena scientifically is the sub-competency 
revealing the greatest decrease in the low level: from 83% to 60% in Biochemistry and from 
92% to 78% in Biotechnology. Even though for Biochemistry the percentages rise for both the 
average (33%) as well as for the maximum (7%) levels, in Biotechnology it only rises for the 
average level (22%). 

e USE: using scientific evidence is the sub-competency for which the percentage of 
students in the low level increases most; in Biochemistry from 30% to 70% and in Biotechnology 
from 32% to 52%. In Biochemistry in particular, the maximum and average levels decrease, 
almost doubling the low level. In Biotechnology the average level shows a slight increase and 
the maximum level decreases strongly (32% to 52%). 

For both courses, the categorical variables that are shown in Table 2, were analyzed. 

In Biochemistry: 

e  TPSC: the level achieved by the students is independent from when this sub- 
competency was assessed (Chi-Square Test p=0.528). The percentages of students in the low 
and average levels are virtually identical at both times. The values vary from 2,1% to 7,4% at 
maximum level, this variation is not statistically significant. 

e IST: the level achieved by the students is independent of when this sub-competency 
was assessed (Chi-Square Test p=0.149). However, at the maximum level, there were 17% of 
students at the pre-class and 37% at the post-class. 

e EPS: the level achieved by the students is independent of when this sub-competency 
was assessed (Chi-Square Test p=0.075). Despite this, at the low level, it appears that at the pre- 
class there were 83% of students and 60% at the post-class. 
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e USE: the level achieved by the students is dependent on when this sub-competency 
was assessed (Chi-Square Test p=0.001). It is necessary to highlight that, at the maximum level 
there were 32% of students at the pre-class and only the 4% at the post-class. 

In Biotechnology: 

e TPSC: the level achieved by the students is dependent of when this sub-competency 
was assessed (Fisher’s exact Test p=0.04). There was a significant decrease in the percentage of 
students in the lowest level at the end of the year, from 97% to 70% and this caused the increase 
in the average level. 

e IST: the level achieved by the students is independent of when this sub-competency 
was assessed (Chi-Square Test p=0.624). 

e EPS: the level achieved by the students is independent of when this sub-competency 
was assessed (Fisher’s exact Test p=0.132). 

e USE: the level achieved by the students is independent of when this sub-competency 
was assessed (Chi-Square Test p=0.117). 


Discussion 


It must be stated that the goals for the study were a) to find a trend of change in spite of 
individual variability, and b) track development — that could be done by taking common paths 
of students (pre, and post but also describing their process of learning in the course). Besides, it 
is difficult to assess competencies only through written tests, therefore it would be necessary to 
assess students’ actions and their self-assessment processes (Brown & Glasner, 2003). 

It should be pointed out that the obtained results couldn’t be corroborated with other 
researchers as no similar work was found. 

The results achieved in this study have led to new questions that will be addressed in 
future research. It is particularly important to analyze in greater detail some of the elements that 
influence the development of each sub-competency, such as, for example, the use of models to 
explain scientific phenomena; the use of scientific evidence to draft an explanation; and which 
features of the experimental activities may foster student involvement to identify scientific is- 
sues. Moreover, the queries that have been performed so far have been in Chemistry classes and 
with the first-year students. In order to be able to discuss in further depth the influence of the 
education that is taught, it must conduct both longitudinal and transversal studies at different 
periods of higher education. 

There is awareness that further research into the assessment and evolution of the stu- 
dents’ science competency could provide a better understanding of students’ difficulties and 
teachers’ resources when developing and improving such competencies. 

Performance could be improved if when designing, implementing and evaluating the 
curriculum more emphasis were given to science competency, with the following priorities: 

e Interpreting scientific evidence and drafting and communicating conclusions. 

e Insisting on qualitative scientific explanations, with emphasis on the application 
of the knowledge of scientific concepts in a given situation, describing or interpreting 
phenomena. 

e Identifying problems that are susceptible of scientific research. Knowing the main 
characteristics of research. Recognizing key words in order to search for information. 

The implications brought forward by this new approach to the work of teacher bodies 
in general and to teaching in particular evidence that the assessment must be consistent with 
the remainder of elements in designing education schemes, and as a consequence, competency- 
based assessments must be incorporated (Gijbels, Van de Watering & Dochy, 2005). 
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Conclusions 


Considering that the students’ initial science competency is different and that the learning 
processes are personal and possibly different for each individual, it cannot be expected that one 
single assessment perspective will be totally effective in the sense that they will all develop and 
express the aforesaid competencies in the same way. However, considering that learning is an 
enrichment of the individual’s cognitive and emotional structure, it can be concluded, taking 
into account the results, that learning is a gradual process and not one of a linear change of 
ideas, and that the evolution of competency is connected to such process. 

Coinciding with the above, in TPSC, both courses show low scores at the beginning of 
the study. Then the results improve very slightly with education. 

In respect of the swb-competencies assessed, in [SJ and EPS, Biochemistry students 
develop and improve such skills with education to a greater extent than does Biotechnology. 

In the sub-competency USE, the students of both courses start off with similar baseline 
values and they do not improve with education, even less in Biochemistry. 

The study practically started with two similar groups and ended in the same way. The 
development of science competency doesn’t depend on education in the first year of the courses 
of Biochemistry and Biotechnology. 
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